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Abstract 

The problem of the minimization of least squares functionals with £^ 
penalties is considered in an infinite dimensional Hilbert space setting. 
While there are several algorithms available in the finite dimensional set- 
ting there are only a few of them which come with a proper convergence 
analysis in the infinite dimensional setting. 

In this work we provide an algorithm from a class which have not been 
considered for £^ minimization before, namely a proximal-point method 
in combination with a projection step. We show that this idea gives a 
simple and easy to implement algorithm. We present experiments which 
indicate that the algorithm may perform better than other algorithms if 
we employ them without any special tricks. Hence, we may conclude that 
the projection proximal-point idea is a promising idea in the context of 
^^-minimization. 

1 Introduction 

In this work, we consider the ^^-minimization optimization problem. Let K : 
£^ H he a bounded linear operator mapping the sequence space £^ into a 
Hilbert space H, g £ H and a > 0. The minimization problem reads as 

min*(M) with '^{u) l\\Ku - g\\jf + a\\u\\i. (1) 

We follow [20] and derive a projection proximal-point algorithm which sequen- 
tially solves a regularized problem 

mmj>{u) + '^\\u^um (2) 

up to desired accuracy and then applies a projection which reduces the distance 
to the minimizer of the original problem (1). While the regularized problem (2) 
is still non-smooth, it turns out that it can be solved easier than the original 
one (1). 

The main aim of this article is, to provide another alternative approach to i^- 
minimization in the infinite dimensional setting. Other approaches use surrogate 
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functionals [7] , proximal forward-backward splitting [6] or generalized gradient 
methods [5, 4]. While all the mentioned approaches lead to the same iterated 
soft-thresholding procedure, other methods use iterated hard-thresholding [3] 
or an active set approach formulated as a scmismooth Newton's method [13]. 
We remark that in finite dimensions several other algorithms are available as, 
e.g., gradient type methods like GPSR [11] or fixed point continuation [14], 
interior point methods [17] or active set methods like LARS and the homo- 
topy method [10, 19, 9] to name just a few. The contribution of the paper is 
hence twofold: one the one hand, we add another class of algorithms, namely a 
proximal-point-like algorithm, to the zoo of available methods and on the other 
hand we provide an algorithm which is globally and strongly convergent in the 
infinite dimensional setting. We stress, that the purpose of this paper is not to 
develop an algorithm which outperforms other existing methods. 

The article is organized as follows: In Section 2 we review the projection 
proximal-point algorithm for the solution of maximally monotone operator equa- 
tions. In Section 3 we show how the subproblcms in the £^ case can be solved by 
iterative thresholding or the generalized conditional gradient method. In Sec- 
tion 4 we state the full algorithm and show linear convergence of the method. 
Section 5 presents numerical experiments and Section 6 concludes the article. 

2 The projection proximal-point algorithm 

In this section wc review briefly the projection proximal-point algorithm from [20] 
Further we show how it can be applied in the context of ^^-minimization. 

2.1 The projection proximal-point algorithm for general 
mctximal monotone inclusions 

The projection proximal-point algorithm has been proposed to solve the fol- 
lowing inclusion problem: Let H be Hilbert space and T a maximal monotone 
operator on H. Find u & H such that 

e T{u). (3) 

The algorithm iteratively solves a regularized subproblem: For a given iterate 
u" and a parameter /x„ > find as an approximate solution of 

oer(u) + Ai„(w-w") (4) 

The notion of "approximate solution" is made precise by saying that is an 
approximate solution of (4) if for small e" it holds 

= t;"+i-hM«K+i-w")-|-e", eT(u"+i). (5) 

After the solution of this approximate problem, a projection step follows which 
provably reduces the distance to the solution set of (3). In general the regu- 
larized subproblem (4) may be as hard as the original problem (3) but it turns 
out that often it is comparably easy to solve. The total algorithm is stated as 
Algorithm 1. 

Step 4 of the algorithm solves the regularized subproblem approximately 
up to a desired accuracy. The accuracy is tuned by the parameter a. The 
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Algorithm 1 Projection proximal-point algorithm 

Require: u° £ H, a € [0, 1[, rimax 
1: Set n <— and done <— false. 
2; while n < rimax and not done do 
3: Choose /Lt„ > 0. 
4: Calculate y" and e" such that 

= t;" + M„(y"-i^") + e" 
e T(2/") 

<'7max{|K|Un||2/"-z*"||}. 

5: if = or = u" then 

6: done <— true 

7: else 

8: Calculate it" as 

9: Set n <— n+ 1. 
10: end if 
11: end while 



parameters /LI„ give the amount of regularization of the subproblem to be solved 
in step 4. Note that step 8 is actually the projection onto the hyper-plane 
S„ = {u G H : {v", u — y'^) = 0}. Note that by the condition on ||e"|| in 
step 4 one concludes that Sn separates the iterate m" from the solution set 
of (3) (see [20, Theorem 2.2]). 

2.2 The projection proximal-point algorithm for ^^-minimization 

Wc apply Algorithm 1 to the minimization problem (1). The maximal monotone 
operator T is given as the subgradient of the objective functional With the 
multivalued sign-function 



Sign(u)fe e < 

this problem reads as 



{!}, Uk>Q 

[-1,1], Uk=Q. 
{-I}, Uk<0 



e K*{Ku- g) + aSign{u). 
It is an easy observation that the regularized subproblem correspoding to (4) is 
e K*{Ku - g) aSign(M) + ^ln{u- (6) 
and is equivalent to the problem 

2 , „,|L,|| . I //„ II •// ||2 



niin ijlifu - fflll^ + a||u||i + ^iiu, - u, 



u — u 
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Hence, the subproblems are regularized by adding the ^^-distance to the previous 
iterate. 

The crucial step in Algorithm 1 is step 4 where one needs to solve the 
regularized problem up to a desired accuracy. In Section 3 wc describe two 
algorithms with are shown to produce solutions with the desired accuracy iter- 
atively. Hence, our algorithms consists of two nested loops. At first glance one 
may think that this will result in bad performance but it tiirns out that the 
inner loop usually terminates quite fast (depending on the choice of /z„ and a). 
Before turning to the full algorithm in the case of ^^-minimization we present 
two algorithms to solve the subproblem in step 4 of Algorithm 1. 

3 Iterative thresholding algorithms for the reg- 
ularized subproblem 

Every iteration of the projection proximal-point algorithm involves the approx- 
imate solution of a regularized problem of the form 

minl\\Ku-gr + a\\uh + ^\\u-um (7) 

In the next subsection wc show, that these subproblems can be solved by means 
of either the a generalized gradient projection method from [4] (which is in 
fact a damped iterative soft-thresholding) or a generalized conditional gradient 
method from [5, 3]. 



3.1 Damped iterative soft-thresholding 

In this subsection we assume that \\K\\ < 1 (a condition which may alwas be 
fulfilled by rcscaling the problem). To derive an algorithm for the approximate 
solution of the regularized problem (7) we use the characterization (6) for a 
solution u: 

G K*{Ku - g) aSign(u) + /i„(u - m"). 
We rewrite the characterization as 

u - K*{Ku -g)+ iJ.„u"- G ((1 + Mn) id+aSign)(u) 

which leads to 

j^{u- K*{Ku -g)+ M„w") e (id+^ Sign)(«). 

Since Sign is a maximal monotone operator (as the subgradient of a proper, 

convex, and lowcr-scmicontinuous fimctional). (id +c Sign) possesses a single 
valued inverse for any c > 0. This can be given explicitly as 

{Uk - C, Uk > c 
0, |Mfe|<C 
Uk + C, Uk < -c 



Hence, u fulfills the fixed-point equation 

T^S^iu - K*{Ku -g)+ finu"). 



u =S^^{j^{u-K*{Ku-g) + ^inu'')) 
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Since the operator Sa is non-expansive and we assumed that < 1, we see 
that the mapping u i-^ (1 + iJin)~^Sa{u — K*{Ku — g) + /Unu") is a contraction 
with constant (1 + /Lt„)~^. By the Banach fixed point theorem we have the 
following result. 

Theorem 3.1. The iterates 

= T^S^iy^ - K*{Ky^ - g) + (8) 

converge with linear rate to the solution of (7), especially it holds for the solution 
y of (7) that 

\\y'+'-y\\<T^\\y'-y\\- 

Remark 3.2 (Damped iterated soft-thresholding as generalized gradient projec- 
tion method). An alternative motivation for the above algorithm is as follows. 
We split the objective function as 

F{u) = l\\Ku-gf, ^u)=a\\u\\i + tf\\u-u"'f. 

Now we apply the generalized gradient projection method from [4] to the prob- 
lem 

mini^(M) +^{u). 

u 

The algorithm is 

= J, J/ - SfeFV)) 

where F'{y'') = K*{Ky'^ — g) and Jg is the proximal mapping 
Js{y) = BxgTxim\\\v - y\\^ + s^{v). 

V 

One easily verifies that 

and hence the generalized gradient projection method gives 



y 



k+l 



-S,,a{y'' - SkK*{Ky'' - g) + ^inu"^) 



which is the same as (8) for Sfe = 1. In [4] it is shown that the generalized 
gradient projection method converges as soon as the stepsizes fulfill < s < 
Sk<s<2/\\Kf. 

Finally we state the damped iterative soft- thresholding as Algorithm 2. 

3.2 Generalized conditional gradient method 

The generalized conditional gradient method as proposed and analyzed in [3, 5, 
1] offers another possibility for the approximate solution of (7). As in Remark 3.2 
we split the objective function as 

F{u) = ^\\Ku - gf, ^u) = a\\u\\, + ^||« - 

Now wc apply the generalized conditional gradient method to minF(?i) + ^(u). 
In the case F{u) = ■^\\Ku — gW^ and a general convex $ the algorithm is given 
in Algorithm 3. 

For our special choice of $ the search direction w'^ in (9) is given by the 
following lemma. 
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Algorithm 2 Dainpc;d iterative soft-thresholding 

Require: y° € H, fcmax, s, s 
1: Set fc ^ 
2: while k < fcmax do 

3: Choose 0<s<Sfe<s< 2/||if IP and update 

= T+^^«.a(y' - SkK*{Ky'' -g)+ f^nU^ 

4: Set n <— n + 1 
5: end while 



Algorithm 3 GeneraHzed conditional gradient method 

Require: y° G H, fcmax 

1: Set A: ^ 

2; while k < fcmax do 

3: Calculate a search direction w'^ as 

w'' G axgmm{K*{Ky'' - g), w) +^{w). (9) 

w 

4: Calculate a step-size Sk according to 

- + {Ky'' - g, K{y'' - w'')) 

WKiy'^-w'^W J ■ 

5; Update 1/'^+-^ = y'' + Sk{w'' — y^) and set n <— n + 1 
6: end while 



Sk = mm 



6 



Lemma 3.3. iei = a||u||i + ^||u — u"||^. The solution of (9) is given by 

w = j^S^iUnvT - K*{Ky^ - g)). 
Proof. A solution w of (9) is characterized by 

QeK* {Ky'' -g) + a Sign(w;) + /x„ (w; - u") 
which we rewrite as 

- ^K*{Ky'' -g)€ (id+^ Sign)(w;). 
Similar to the calculation in Section 3.1 this leads to 

w = j^SaifinU^ - K*{Ky'' - g)). 

□ 

Using techniques from [3] we derive the following result: 

Theorem 3.4. Let^{u) = a\\u\\i + ^\\u — Then the iterates y'^ produced 

by Algorithm 3 converge with a linear rate to the unique solution of (7). 

Proof. We use Theorem 7 from [3]. Let y* denote the unique solution of (7). 

In [3, Theorem 7] it is shown, that one gets linear convergence of the iterates of 
the generalized conditional gradient method as soon as an estimate 

{K*{Ky* -g),v- y*) + ^{v) - > c\\v - y*f (10) 

holds locally around y*. Since y* is a solution of (7) we conclude that 

-K*{Ky* -g)&d^{y*). 

and hence, the right hand side of (10) a Bregman distance with respect to By 
standard argument from convex analysis we conclude that the sub differential of 

$ at y* is a Sign(y*) + Hn{y* — w") and hence we have 

-K*{Ky* - g) = aw + /U„(y* - -u") with w e Sign(y*). 
Now we estimate 

{K*{Ky* -g),v- y*) + ^v) - $(y*) 
= J2 -(^^i + l^niy* - umvi -y*) + a{\vi \-\y*\) + l^{{v,- u^f - [y* - u^f) 

i 

= (b,! - \yl\ - wM - VD) +^{{vi-Kf-{yl-Kf-2{yl-u-){v,-yl)) 

i ^ ^ ' 

>o 

i 

i 

Hence, we proved (10) with c = /U„/2 and the claim follows from [3, Theorem 7] 

□ 
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4 Full algorithm and convergence properties 



In the previous section we derived two algorithms which solve the problem (6) 
which is needed in step 4 of Algorithm 1. For a given m" both algorithms 
produce iterates which converge with a linear rate to the solution of (6). 
Hence, it remains to check when to stop the iteration to fulfill the condition in 
step 4 of Algorithm 1, namely: 

e r(/) (11) 

= w" + ^„(?/'= -u") + e" (12) 

<c7ma^{|K||,M„||2/'=-w"||}. (13) 

To do so, wc proceed as follows: Given an iterate of the outer iteration 7i" and 
an iterate of the corresponding inner iteration we define the projection onto 
the set aSign(y'^) as 



-PaSigii(j/'=)('")i 



a > or {y^ = and u, > a) 

—a < or {y^ = and Uj < —a) 
Ui = and \ui\ < a. 



Then we calculate 

= (id -P„signto^))(-^*(i^y' -9)- - 

and 



It is obvious that then (12) is fulfilled. Moreover one sees 

v'^ = ~Ai„(/ - u^) ~ {id-P^sisn(y^))i-K*{Ky' -g)- M„(y'= - «")) 
= K*(Ky'' - g) + P„sig„fo'=)(-i^*(i^/ - 5) - - 
e K*{Ky>' -g)+a Sign(y'=) = T(/). 

Finally, it remains to check the inequality (13) to accept an iterate y^ . 

Putting the pieces together, we get the projection proximal-point algorithm 
for ^^-minimization as Algorithm 4. 

The projection proximal-point algorithm is known to converge Q-linearly 
under certain requirements. Wc cite from [20]: 

Theorem 4.1. Let T he maximally monotone, let u he a solution of (3) and let 
be bounded from above. If moreover there exists a constant L > such that 
for all y, v with \\v\\ < 5 and such that v G T{y) it holds 

\\y-u\\ < L\\v\\ 

then the projection proximal-point algorithm 1 converges Q-linearly. 

In the special case of f ^-minimization the above theorem is applicable if the 

operator K obeys the finite basis injectivity property from [4]: 

Definition 4.2. An operator K : £^ ^ H mapping into a Hilbert space has the 
finite basis injectivity (FBI) property, if for all finite subsets / c N the operator 
K\i is injective, i.e. for all u,v € i'^ with Ku = Kv and Uk = Vk = for all 
k ^ I \i follows u = V. 
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Algorithm 4 Projection proximal-point algorithm for ^^-minimization 
Require: u° £ H, a € [0, 1[, nmax 
Set n <— and doneouter ^ false, 
while n < rimax and not donCouter do 
Choose /z„ > 0, and 
Set k -i— 1 and doncinner <— false, 
while not doneinner do 

Calculate y'^ via one step of either Algorithm 2 or 3. 
Calculate 

e'= = (id-P,sig„(,'=))(-i^*(^2/' -9)- - 
if lle'^ll < amax{||i;'=||,/x„||/ - then 

donCinner ■*— trUC 

else 

Set fc ^ A:-M. 
end if 
end while 

if ii" = or J/" = m" then 

doneouter <— true 
else 

Calculate m" as 



Set n <— n -|- 1. 
end if 
end while 



9 



Theorem 4.3. Let K : £^ ^ H obey the FBI property. Then Algorithm 4 for 

£^ -minimization converges globally and Q -linearly. 

Proof. We show that the conditions in Theorem 4.1 are fulfilled. Hence, we 
consider T{u) = K*{Ku - g) + a Sign('«) and e T{u) and v e T{y). 

First we show, that w ^ implies y ^ u. To this end, we remark that y 
solves 

y € argmin4'(M) — (u, v) . 

u 

We now show, that the functional \E'(u) — (u, v) F-converges to Consider a 
sequence {vj) in ^ with and define 

*i(w) = ^(«)-(«j, w). 

Since \I' is lower semi-continuous it holds for every Uj — > u that 

liminf \E'j(uj) = liminf ^'(uj) — Vj) > \E'(u). 
j 3 

Moreover, for the constant sequence Uj = u we see that 

limsup^'j(uj) = limsup^(u) — {u, vj) = ^'(u) 

3 3 

and hence, F-converges to ^ (see, e.g. [2]) and in particular the minimizers of 
converge to that of i.e. for yj such that vj S T{yj) it holds that yj u. In 

other words: For e > there exists S > such that < S implies ||y — it|| < e. 
For some s > the quantities m, u, y are characterized by 

u = Sa{u-K*{Ku-g)) and y = Sc,{y - K*{Ky - g) + v). 

Similar to [13, Proposition 3.10] one sees, that for — u|| < e and ||w|| < S 
small enough there exists a number fco such that 

{fcGN I \y-sK*{Ky-g)+v\k<sa}c{l,...,ko}=:I. 

This shows, that the supports y are uniformly bounded in a neighborhood of u. 
Moreover, for < s < 2/||i^p it holds 

\\u - y\\ = \\S^{u - K*{Ku - g)) - Sa{y - K\Ky - 5) + v)\\ 
< \\{iA-K*K){u-y) + v\\ 
<\\{:,A-K*K){u-y)\\ + \\v\\ 

Since K obeys the FBI property and suppy,suppu C /, there exists c 6]0, 1[ 
such that ||(id —K*K){u — y)|| < c\\u — y\\ and we finally conclude 

< Yz-^ll^ll- 

The Q-linear convergence now follows from Theorem 4.1. □ 

5 Numerical experiments 

In this section we present example calculations to illustrate how the algorithm 
works in different settings. 
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Figure 1: Left: Convolution kernel K for digital holography A = 630nm and 
recording distance of 250nim. Right: Simulated hologram with 20 particles. 



5.1 Digital holography 

As a first example problem we consider the problem of digital holography. In 
digital holography, the data correspond to the diffraction patterns of the ob- 
jects [12, 18]. Under Frcsncl's approximation, diffraction can be modeled by 
a convolution with a "chirp" kernel. In the context of holograms of parti- 
cles [23, 15, 16], the objects can be considered opaque (i.e., binary) and the 
hologram recorded on the camera corresponds to the convolution of disks with 
Presnel's chirp kernels. The measurement of particle size and location therefore 
amounts to an inverse problem [22, 21, 8]. 

Wc consider the problem of locating small objects (points), i.e. we assume 
that opaque objects are distributed in three-dimensional space for which we 
use the coordinates {x,y,z). For objects which are located in the plane Zj and 
an icident laser beam in — z-direction of wavelength A, the amplitude in the 
observation plane at z = is well modeled by a bidimensional convolution with 
respect to the variables {x, y) with the Presnel function 



However, one is only able to measure the absolute value and not the com- 
plex valued amplitude. After simplification, the problem of reconstruction of a 
diffraction pattern from objects in the z^-plane can be modelled as deconvolu- 
tion with the following kernel, see Figure 1 



We generated a hologram g with a number of particles all located in the 
same plane. ^ Then we applied the projection proximal-point algorithm to the 
solution of the inverse problem with the operator 



We show reconstruction of the objects with different methods in Figure 2. Note 
that all methods delivered well seperated and moderately sharp objects. How- 
ever, the result for the projection proximal-point algorithm with generalized 
conditional gradient method gives a slightly sharper reconstruction. 

^Thc author would like to thank Loi'c Denis (Ecolc Supcricurc dc Chimie Physique 
Electronique de Lyon) for providing the implementation of the hologram simulator. 




{x, y) = h^^ (x, y) = 



Ku = Kzj * u. 
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ppp, soft thresholding soft thresholding 



ppp, gen. cond. grad hard thresholding 

Figure 2: Reconstructions of the hologram from Figure 1 with different methods 
and 120 iterations. Top left: Projection proximal-point method with damped 
soft thresholding in the inner loop. Top right: Iterated soft thresholding [7]. 
Bottom left: Projection proximal-point method with generalized conditional 
gradient method (Algorithm 3) in the inner loop. Bottom right: Iterated hard 
thresholding [3]. 

In Figure 3 the development of the functional value is shown. We remark, 
that the projection proximal-point algorithm does not necessarily reduce the 
functional value and sometimes, depending on the problem and the parameters, 
does not produce monotonically decaying objective values. However, this may 
be an advantage of the algorithms since it does not have to follow a decent di- 
rection. Note that iterated hard-thresholding (which is also a generalized condi- 
tional gradient method) performs worst and that the iterated soft-thresholding 
alone or in combination with the proximal-point modification behave better 
and somewhat similar. However, combining the generalized conditional gradi- 
ent method with the proximal-point modification gives significant improvement. 

5.2 Deblurring and the influence of fi and a 

The parameters /x and a influence the behavior of the algorithm. BasicalUy, the 
parameter /i influences how much the subproblcms are regularized and hence, 
for larger values of /x the subproblcms are solved faster, sec, e.g. the rate of 
convergence in Theorem 3.1. The parameter a tunes the desired accuracy for 
the subproblcms, i.e. the smaller a is, the more accurate is the solution of the 
subproblem and hence, the subproblcms are solved slower. As a rule of thumb 
one may remember that it does not seem necessary to choose a too small, i.e. it 
is enough to solve the subproblcms only roughly. Typically a = 0.9 is a good 
choice. On the other hand, not too much regularization is necessary to terminate 
the inner loop quickly, i.e. small values of jj. give good results. A typical value 
for may he fj, = 0.05. Moreover, smaller fj, often lead to faster decay of the 
functional value in the experiments. 
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100 



Figure 3: Decay of the functional value for the problem of digital holoraphy 
example and different methods. The parameters are similar toFigure 2. 




500 1000 



Figure 4: Decay of the functional value for the projection proximal-point algo- 
rithtm with soft-thresholding in the inner loop for the deconvolution problem 
and different values of a and /i. 

To illustrate this behaovior we performed a one dimensional deconvolution 
experiment. We considered a discretized linear blurring operator A which con- 
sisted of the circular convolution with the kernel k,{x) = 1/(1 -l-cc^/S^) and com- 
binded this with a synthesis operator B associated with simple hat-functions 
and hence, considered the operator K = AB. We generated data which just 
consists of a few spikes and hence, has a sparse representation in hat functions. 
We ran the projection proximal-point algorithm (Algorithm 4) with different 
values for fj, and a and with both soft-thresholding (Algorithm 2) and the gen- 
eralized conditional gradient method (Algorithm 3) in the inner loop. The decay 
of the objective value as shown in Figure 4 and 5 is typical. 

To get an impression, how the values of the parameter /x and a influence 
the termination of the inner loop due to conditons (11)-(13), we recorded the 
numbers of inner and outer iterations for different values of /j, and a in Table 1 for 
soft thresholding and in Table 2 for the generalized conditional gradient method. 
In all cases we ran the algorithm until 350 total iterations have been done. (Note 
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Figure 5: Decay of the functional value for the projection proximal-point algo- 
rithm with the generalized conditional gradient method in the inner loop for 
the deconvolution problem and different values of a and \x. 



that this sometimes causes that the last inner iteration did not terminate, see 
e.g. Table 1 bottom left). First one notes that smaller values of a cause the inner 
iteration to take longer to terminate and the same holds true for smaller values 
of /Lt. Moreover, we see that smaller values of a (i.e. solving the subproblems 
more precise) does not improve the decay of the functional value. Hence, we 
may conclude, that a larger value of a is preferable in general. The value 
of [I behaves differently for soft-thresholding and the generalized conditional 
gradient method. While for the first, smaller values of /i lead to better results 
concerning the functional value, for the latter the converse holds true. Finally, 
the generalized conditional gradient method gives in general smaller functional 
values. 



6 Conclusion 

The projection proximal point algorithm as proposed in [20] is applicable for the 
problem of ^^-minimization. It could be shown that under the FBI assumption 
the algorithm even converges linearly. Both the iterated soft-thresholding and 
the generalized conditional gradient method are applicable methods to solve 
the regularized subproblems. For the iterated soft-thresholding the projection 
proximal-point algorithm does not lead to a significant improvement. On the 
theoretical side, the iterated soft-thresholding itself converges linearly [4, 14] and 
on the practical side, both methods behave comparable (see Figure 3). However, 
for the generalized conditional gradient method the situation is different. For 
the algorithm itself as stated in [3] the distance to the minimizer only converges 
like C(n~^/^) but the projection proximal-point extension gives linear conver- 
gence. Moreover, also the practical behavior is better, see Figure 3. In total, it 
seems that the generalized conditional gradient method combines well with the 
projection proximal point idea. 

However, the aim of this paper was not to develop the fastest available 
method for ^^-minimization but to provide an algorithm from a different class 
of methods. Hence, it may be expected that after considerable fine tuning 
(parameter choice rules, more efficient solvers for the subproblems, backtrack- 
ing,...) the projection proximal-point algorithm will become comparable to 
state-of-the-art methods. 
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^ = 0.2, a = 0.1 ^ = 0.2, a = 0.9 



# out iter 


# in inter 






# out iter 


# in inter 




1 


6 


5 


.03e-02 


1 


1 


8.88e-02 


2 


8 


1 


.91e-02 


2 


1 


5.20e-02 


3 


9 


1 


.21e-02 


3 


1 


3.68e-02 


31 


11 


2 


. 13e-03 


347 


1 


1.46e-03 


32 


11 


2 


. lOe-03 


348 


1 


1.46e-03 


33 


11 


2 


.06e-03 


349 


1 


1.46e-03 


34 


11 


2 


.03e-03 


350 


1 


1 .46e-03 


M 


= 0.01, a = 


0.1 




M 


= 0.01, a = 


0.9 










# out iter 


# in inter 


*(m") 










1 


6 


2.08e-01 










2 


11 


1.49e-01 


# out iter 


# in inter 




*(w") 


3 


12 


9 . 76e-02 


1 


101 


5 


.06e-03 








2 


190 


2 


.06e-03 


16 


14 


1.70e-03 










17 


14 


1.63e-03 










18 


13 


1 . 58e-03 










19 


13 


1 . 54e-03 



Table 1: Output of the projection proximal-point algorithm with soft thresh- 
olding in the inner loop. The problem under consideration in the deblurring 
problem. 
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= 0.2, a = 


0.1 






= 0.2, a = 0.! 


3 




# out iter 


# in inter 






# out iter 


# in inter 






1 


5 


4 


.94e-02 


1 


1 


1 


.54e-01 


2 


5 


1 


.92e-02 


2 


1 


7 


.78e-02 


3 


7 


1 


. 18e-02 


3 


1 


4 


.36e-02 


117 


2 


1 


.27e-03 


347 


1 


1 


. lOe-03 


118 


6 


1 


.27e-03 


348 


1 


1 


. lOe-03 


119 


1 


1 


.26e-03 


349 


1 


1 


. lOe-03 


120 


1 


1 


.26e-03 


350 


1 


1 


. lOe-03 


M = 


= 0.01, a = 


0.1 






= 0.01, a = 0.9 




# out iter 


# in inter 






# out iter 


# in inter 




*(m") 


1 


27 


4 


.59e-03 


1 


5 


1 


.84e-01 


2 


32 


1 


.97e-03 


2 


4 


5 


.94e-02 


3 


85 


1 


.60e-03 


3 


4 


2 


.51e-02 


4 


83 


1 


.44e-03 










5 


1 


1 


.33e-03 


56 


3 


1 


.03e-03 


6 


12 


1 


.27e-03 


57 


6 


1 


.05e-03 


7 


97 


1 


.22e-03 


58 


44 


1 


. 02e-03 


8 


1 


1 


. 19e-03 


59 


1 


1 


.Ole-03 



Table 2: Output of the projection proximal-point algorithm with the generalized 
conditional gradient method in the inner loop. The problem under consideration 
in the deblurring problem. 
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Finally wc remark, that Algorithm 4 docs not rest upon linearity of the op- 
erator and may also be applied to non-linear operators. Also for the solution of 
the subproblems either iterated soft-thresholding [1] or the generalized condition 
gradient method may be used. 
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